Validation and verification apparatus and method

ABSTRACT

Apparatus and a method are disclosed for automatically identifying and validating a document, for validating the identity of a bearer of a document, for verifying that the bearer has authorization to participate in an activity represented by the document, and for comparing information on the document against information databases to determine if there are known concerns about the document or its bearer. A document type is narrowed by initially determining the size of the document, and is then identified from amongst a group of documents by looking for colors or other characteristics at specific locations on the document only until the document is identified. The order in which document locations are looked at is dependent upon a history of the types of documents that have been identified. An identified document is verified by first looking up reference information about the document type and using that information to read specific contents and check only specific security and other features utilized on the document. The contents of a document are checked against a database to verify its contents, and other databases are checked for stolen documents, known valid documents, and prohibited travel or entry. Biometrics in databases, on documents and directly from the document bearer are compared to verify the bearer.

This is a continuation of application Ser. No. 09/994,399, filed Nov.26, 2001 now abandoned.

FIELD OF THE INVENTION

This invention relates to apparatus and a method for automaticallyidentifying and validating a document, for validating the identity of abearer of a document, for verifying that the bearer has authorization toparticipate in an activity represented by the document, and forcomparing information on the document against information databases todetermine if there are known concerns about the document or its bearer.

BACKGROUND OF THE INVENTION

Over time there has been an increase in counterfeiting and alterationsto identity, travel and other documents, such as drivers licenses,passports, identification cards, and passes. In addition, there has beenan increase in counterfeiting and alterations to documents having value,such as negotiable instruments, bonds and other similar documents. Thishas caused concern to the governments and companies who issue suchidentity and travel documents; and much financial loss where suchcounterfeit and altered documents are documents of value.

To hinder such counterfeiting and alterations to identity, travel andsimilar documents, and documents having value, many innovations havebeen proposed or introduced. One solution has been the development andimplementation of new materials for producing such documents that hasmade counterfeiting and alterations more difficult, and the detection ofcounterfeit and altered documents easier and faster. Such new materialsinclude the use of holograms and retro-reflective layers in laminatingmaterial, invisible information that only appears when illuminated bycertain wavelengths of invisible light or other energy, and differenttypes of inks that are seen as one color under normal ambient light butare seen as a different color when illuminated by certain wavelengths ofinvisible light or other energy (chemical taggants). In addition,magnetic and radio frequency (RF) taggants that are invisible to the eyeare added to base materials and laminating materials but may be detectedusing special equipment. Further, micro-miniature smart chips and memorychips are embedded in such documents, just as they are in smart cards,and may be used to identify, read and validate documents in which theyare embedded, and to identify and validate the bearer of such documents.

One example of a security laminating material used foranti-counterfeiting of passports is 3M's Confirm® security laminatedescribed in U.S. Pat. No. 5,658,411. Another example of a 3M securitylaminating material used for anti-counterfeiting of passports isdescribed in U.S. Pat. No. 5,631,064 and utilizes retro-reflective glassmicrospheres.

An example of an identity card using smart-card technology has recentlybeen introduced in Malaysia where an embedded computer chip and memoryallows the card to be used as a combination identity card, driver'slicense, cash card, national health service card, and passport.

Coupled with the increase of new materials and new techniques to producedocuments that are more difficult to counterfeit or alter, there hasbeen an increase in the demand for new equipment and systems forautomatically identifying and validating documents, for validating theidentity of a bearer of a document, for verifying that the bearer hasauthorization to participate in an activity represented by the document,and for comparing information on the document against informationdatabases to determine if there are known concerns about the document orits bearer. This demand has risen because it has become virtuallyimpossible for a person, by themself, to analyze and validate documentsusing such new materials and other techniques.

SUMMARY OF THE INVENTION

The present invention provides new apparatus and a method forautomatically identifying and validating documents, for validating theidentity of a bearer of a document, for verifying that the bearer hasauthorization to participate in an activity represented by the document,and for comparing information on the document against informationdatabases to determine if there are concerns about the document or itsbearer.

The use of the new apparatus and methods speeds the process of checkingdocuments, and achieving more accurate verification of a document andits bearer because individual examiners cannot be expected to rememberor know all information concerning a plurality of document types. Inaddition, individuals get tired, distracted or careless and they makemistakes.

The new apparatus and method automatically identifies and authenticatesexisting, issued documents of different kinds, by checking the intrinsiccharacteristics of the document and its security features to verify thatthe documents are not counterfeited and have not been altered. Thedocument's properties may also be used to query the authority thatissued the document to confirm its validity.

In addition to identification and validation of the document,information contained in the document may be retrieved and used tovalidate the identity of a bearer of the document. Also, the currentstatus of a bearer's authorization to engage in a particular activityindicated by the document may be verified, such as to cross a border,check-in to a flight, cash a check, rent a car, purchase a controlledsubstance, buy hazardous materials, or to enter a secure area. Further,a photo on the document may be captured with sufficient quality to beused for facial matching by machine or human. A picture, signature,fingerprint, iris scan or other biometric information stored on thedocument may be compared to biometric information received directly fromthe bearer of the document, and/or compared to biometric informationretrieved from a central database. Also, the information obtained fromthe document and the bearer of the document may be checked againstinformation stored in other local or distributed databases, such as“watch” lists, “wanted” lists, and prohibited entry lists. In thismanner, both false identities and identity theft are detected. Thecertainty of detection then becomes a major deterrent to such crimes.

Finally, real-time auditing and reporting is done relative to theactivity that is taking place and all information required to assess therisk of individual patterns of behavior. This information may be storedin a database and used by specific authorities as related to theirjurisdictional responsibilities. Protection of the privacy of anindividual is protected unless the bearer voluntarily forgoes suchprotection for added security or as an exception process in the case ofa missing document. If there is sufficient reason to suspect illegalactivity or the strong likelihood of a future threat, then theappropriate law enforcement agencies may over-ride the normal privacyprotection features.

There are several steps involved in implementing the processes describedin the previous paragraphs.

The first critical step is to identify the type of document presented.Several examples of documents that can be identified are passports,visas, driver licenses, alien residence cards, and INSPASS cards. Thefirst step in the process uses an image of the presented document todetermine the physical size of the document. All documents that may beprocessed are classified into a plurality of physical size ranges. Upondetermining the size of the document from the first image the documenttype is thereby narrowed down to a smaller set of documents. To furthernarrow the search the presence of information at specific locations,i.e. test regions, on the document is looked for. For example, thepresence of a certain pattern of colors in the test regions is lookedfor. Using the information found in the test regions on the document thetype of document is identified with a high degree of probability.

To speed up document identity analysis, statistical information iscollected and used to determine the order in which the test regions on adocument are to be checked. For example, if the apparatus is primarilyused to check driver licenses and identity cards in a state liquor storein the State of New Hampshire, the document most likely inserted intothe apparatus for verification will be New Hampshire drivers licensesand identity cards. Accordingly, as the apparatus is in use, over time,it collects information that most of the time New Hampshire driverlicenses and identity cards are verified. Using this statisticalinformation, after a document size range determination is made and thesize range includes New Hampshire drivers licenses and identity cards,subsequent checking for the presence of information at test regions on adocument will initially look for the presence of such information thatis found on New Hampshire documents. Thus, the processing time istypically shortened.

In the second step, using the identified document type, referenceinformation for the document are retrieved. This criterion includesdocument layout, security features and intrinsic characteristics(documetric) for the identified document type. Using the referenceinformation, data and images are collected from test regions on thedocument. Some of the data collected will be obtained usingoptical/intelligent character reading technology and bar-code readingtechnology. Data and images collected from the document are from areasintended for machine-readability; from human readable areas protectedfrom forgery and tampering and not intended for machine readability; andfrom areas that are covert and not visible to the unaided human eye.Images are collected from documents using a variety of excitationsources, including infrared, visible and ultraviolet light sources, andpolarized light based on the known properties for the specific documenttype as found in the reference information. Other information may becollected using devices such as ultrasonic, RF or magnetic sensors.

In the third step, the information and images collected from a documentare analyzed and compared with known characteristics for the identifieddocument type in the reference information. Images are processed tocheck the composition of the document, characteristics of the materials,and evidence of any alteration or counterfeiting. Also, classes ofdocuments with known forgery characteristics are specifically examinedfor the forgery characteristics. At the conclusion of this third step,certain conclusions can be made as to the authenticity of the presenteddocument and a “score” representing the quality of the match between thedocument and the known properties for that document is calculated. Abovea predetermined “score” the document is considered to be a real documentin view of the level of analysis performed. Below a predetermined“score” it can conclusively be determined that the document is notvalid. Between these scores there is questionable authenticity, and thedocument bearer and document may be subjected to closer examination.

In the fourth step the data and image information captured from eachdocument may be used to query relevant databases controlled by theissuing authority of the document, or by agencies with lists of knownstolen documents, watch lists of wanted or suspected individuals,biometric data for identity matching, data combinations such as arrivaland departure locations, suspected combinations of travelers and travelpatterns, known issued valid documents, and other pertinent information.

In the fifth step, after the presented document has been validated, thenidentity verification of the bearer is accomplished linking the bearerto the document. This is done by matching biometric informationcontained on the document with information measured from the bearer ofthe document. If the quality of a biometric on the document (such as aphoto) and a biometric obtained directly from the bearer at theapparatus are sufficient for matching with a high degree of confidence,then no further examination is required. Should the biometric not matchwith sufficient confidence, a further match could be made by using thebiometric information captured to query the authority that issued thedocument for their assessment of the match.

DESCRIPTION OF THE DRAWING

The invention will be better understood upon reading the followingDetailed Description in conjunction with the drawing in which:

FIG. 1 is a block diagram of the novel validation and verificationapparatus used to identify, read and verify documents and their bearers;

FIGS. 2A & 2B are a block diagram showing the steps involved indetermining a document type;

FIG. 3 is a block diagram showing the steps involved in reading andcapturing data from a document and determining if the document iscounterfeit or has been altered;

FIG. 4 is a block diagram showing the steps involved in comparing dataon a document with reference data and data in databases, and determiningif the document bearer is on a watchlist; and

FIG. 5 is a block diagram showing the steps involved in retrieving andcomparing biometric data on a document, in a database and directly fromthe document bearer.

DETAILED DESCRIPTION

In this Detailed Description the operation of our novel validation andverification apparatus 10 in FIG. 1 is described for reading andverifying all types of documents, but with emphasis given to reading andverifying a passport 11. In the description the terms document andpassport are both used, often interchangeably. The detailed descriptionuses a passport by way of illustration, but the same apparatus will alsoread the other types of documents using the same process as described.Other types of documents may be read and verified such as, but notlimited to, driver licenses, identity cards, alien residence cards, andINSPASS cards. While documents of value may also be read and verifiedusing the teaching of the preferred embodiment of the inventiondescribed herein, such documents of value are not specifically describedin detail hereinafter.

On the inside of a passport 11 is located a photograph, bibliographicand possibly other information about the bearer of the passport. This iscalled the “personalization page.” The information on this page includespassport number, issuance and expiration dates, issuing authority,possibly biometric information about the person to whom the passport 11is issued, and other information.

A piece of thin plastic is laminated to the surface of thepersonalization page of passport 11 to seal the photograph andinformation recorded thereon. This plastic laminating material maycontain security features such as holograms or retro-reflectivematerials to provide a measure of security. Laminating, with or withoutsuch security features, is done to make it very difficult to alter apassport or to produce counterfeit passports, but passports are stillaltered and counterfeited despite security measures being taken. Othersecurity measures include affixing holographic seals to the documentunder the laminating layer, using laminating material that has invisibleimages therein, use of special inks, use of special paper and imbeddedthreads, and using paper with a subdued background pattern that isdamaged by attempts to alter the passport.

To make it harder to alter or counterfeit documents, new securitymeasures are continually being developed to make it more difficult, ifnot impossible, to alter or counterfeit documents. Over time such newmeasures have included the use of chemical taggants to produce differentpapers, threads, and types of inks to imprint information. Thesetaggants show up differently under lights of different color, includinginvisible light such as ultraviolet and infrared. These taggants mayalso be magnetic or have other properties that are not apparent, butwhich are detectable when using the proper excitation source and sensor.Further, alterations made to documents containing such securitymaterials, that may be invisible to the naked eye, appear when excitedwith certain wavelengths of light or electromagnetic energy. Stillfurther, special paper may be utilized that is not visibly damaged byattempts to alter the passport, but the damage can be detected usingcertain wavelengths of visible and invisible light, or other techniquessuch as RF or ultrasonic detection.

In FIG. 1 is shown a block diagram of our novel validation andverification apparatus 10. Apparatus 10 has a glass platen 12 onto whicha document 11, such as a passport, is placed. The size and shape ofplaten 12 is chosen according to the intended use of apparatus 10 toaccommodate a variety of different types of identity cards, traveldocuments, resident alien green cards, and other similar documents, suchas drivers licenses, passports, identification cards, ingress/egresspasses; and documents of value, such as bonds, certificates andnegotiable instruments. The CPU 14 and associated peripheral devicesillustrate a fully integrated embodiment of the apparatus. The physicalapparatus may be constructed including only the elements associated withthe camera 18, controller 15, lights 16, optics 17, display 21 andplaten 12. In this configuration the apparatus serves as a peripheral toa processor. However, all of the functionality and process steps remainthe same.

Lights 16 comprise a plurality of visible and invisible light sources orother excitation means of different wavelengths. These sources include,but are not limited to, visible light, near infrared (IR), long andshort wave ultraviolet (UV), from arrays of light emitting diodes(LEDs), RF and ultrasonic, solid-state emitters, transducers, gasdischarge, incandescent, and/or fluorescent sources. The emission fromeach of which may pass through a diffuser medium or focusing mechanism(not shown) to illuminate the page of document 11 on platen 12. Aspecial instance of these lights 16 uses near-infrared (IR) and bluelight from light emitting diodes to emulate a point source of light andilluminate a laminated page of the document. Such illumination is donecoaxially with the path the reflected light travels to camera 18. Camera18 has an operational frequency range that is able to image 400nanometer to 1 micron, visible to near-IR. It can also be enhanced withtransluminance materials to shift the response range to detectfrequencies outside this range. The capabilities of camera 18, as shown,may also be extended to add filters or include transducers sensitive tofrequencies in any range.

In addition, all light sources have programmable output intensities andLED sources may be pulsed to achieve higher peak power levels thatprovide greater illumination of the document and to help to exposesecurity markings and unauthorized alterations at different levelswithin the passport. The frequency of pulsing the IR and blue light LEDsis high enough that the pulsing cannot be detected by camera 18. Thelight from the sequentially energized multiple light sources 16 isreflected from the laminated page of passport 11 on platen 12 andimpinges on optics 17 which focuses the image for camera 18. The optics17 can also be used to split the optical path to support multiplecameras/detectors and/or provide filtering to band limit the spectralcontent reaching the camera.

In this embodiment camera 18 is a high-resolution, color, solid-statecamera that is controlled by CPU 14, and produces a digitized colorpicture. Camera 18 and near-IR light 16 are continuously operating whileapparatus 10 is powered up and awaiting detection of a document. Whenapparatus 10 is to commence functioning, the presence of a document 11on platen 12 is imaged by camera 18 and a first picture is sent to CPU14 via an interface link 27. Link 27 is a high-speed, digital interfacesuch as conforms to the IEEE 1394 standard, high-speed Ethernet, or auniversal serial bus (USB), or other future protocols. The color pictureis in a digitized format and CPU 14 stores it in a memory queue. In someinstances gray scale pictures may be utilized. In other applicationswhere color is not needed a monochrome camera may be utilized.

A local high capacity storage device 25 may store information such asdocument reference information files and image reference files, and aprogram run by CPU 14 to control the operation of validation andverification apparatus 10. Alternately, document reference informationfiles and image reference files and other material may be filed in andaccessed from a remote server. As the capability to check new documenttypes is added to apparatus 10, and new and/or updated information aboutexisting document types or security checks becomes available, that newor updated information is stored on storage device 25 from either adetachable CDROM drive 26 or from a remote server (not shown) vianetwork port 22. Like all computers, there is memory 20 that comprisesstatic and dynamic memory and it functions with CPU 14 in a well-knownmanner. In addition, network port 22 can be used to connect apparatus 10to a server on a local area network (LAN) or wide area network (WAN) torecord the time and date that the bearer of a passport is entering orleaving a country, and transfer other information retrieved from adocument, and to communicate with authorities responsible for databasesused for verifying and validating the document and its bearer, and tocheck if the bearer of the document is on a prohibited entry or wantedlist (“watch list”).

Apparatus 10 also includes a display 21, which may include atouch-screen for response, to provide indications to the either theoperator of apparatus 10 or to a person whose document is being checked,such as the validity or invalidity of document 11, validating orinvalidating the identity of a bearer of the document, verifying or notverifying that the bearer has authorization to travel using thedocument, and indicating detection of an altered or counterfeitdocument. A separate display may also be provided for persons whosedocuments are being checked. There is also a display monitor 24 that maybe used for a number of purposes including displaying to an operatormore detailed information generated by apparatus 10, and pictures andother information retrieved from a remote network of computers vianetwork port 22. In addition, there may be a keyboard 23 that is used bythe operator to manually input information to apparatus 10.

In some applications apparatus 10 is in a stand-alone operation, such asat a point of ingress or egress, and a bearer of a document 11 places iton apparatus 10 to be verified and granted permission to enter ordepart. If document 11 is not verified a visual and/or audibleindication may be provided to the bearer of a document 11 to take someaction, such as reporting to some other location for verification.

Other apparatus may be provided as required by specific applications,such as a bar code reader, a magnetic stripe reader, iris reader, asignature capture device, gate controller, and/or a smart card reader,all of which are not shown in FIG. 1.

In FIG. 2 is a general block diagram showing the program steps involvedin determining a document type. At block 30 it is determined if there isa document 11 on platen 12. Camera 18 and a light 16 (IR) remain on todetect the presence of a document on platen 12 by a change of thedigital image output from the camera. A light 16 (visible) is energizeduntil a first image is captured of document 11. While the presence ofdocument 11 is not detected the program exits block 30 at NO andcontinuously cycles back to the input of block 30. Once something isdetected on platen 12 the program exits block 30 at YES and progressesto block 31 where the first image is captured for processing todetermine what type of document is on platen 12.

At block 32, the first picture is analyzed to locate its edges and fromthis the document size is determined. All documents that may beprocessed by apparatus 10 have a physical size and apparatus 10determines the document size to be within one of a plurality ofpre-selected size ranges, and there is a specific set of test regionsand characteristics for each stored for the documents in each sizerange.

Upon determining the physical size of document 10 from the first image,the physical size range in which the document is located is identified,and apparatus 10 has thereby narrowed down the document to be one of asmaller set of documents. To further narrow the search, the presence ofinformation at specific test regions on the document are looked for. Forexample, the presence of certain color patterns in specific testregions. The information found at the specific test regions on thedocument is used to identify the type of document. If, within theaforementioned smaller set of document types, there is only a singledocument type that has specific information present at a test region,and the information is actually present at the test region in the firstpicture, the document type is ascertained with a very high degree ofconfidence and there is no need to check for the presence of informationat the other test regions on the document. However, if a few documenttypes have similar information present at a specific test region, thenumber of possibilities of the document type has been narrowed, but thedocument type has not yet been ascertained. Thus, the presence of secondspecific information is looked for in a second test region in the firstpicture. If there is only a single document type that has the secondspecific information present at the second test region, and theinformation is actually detected in the second test region, the documenttype is ascertained with a very high degree of confidence and there isno need to check for the presence of additional information in the firstpicture. Only if necessary, this process may be repeated for morespecific information at additional test regions on the document, but theprocess is terminated as soon as the document type is ascertained with avery high degree of confidence. In this manner, testing to determine adocument type is kept to a minimum.

At block 33 a set of characteristics for the ascertained document sizerange are read from the master document file. At block 34, a descriptionof the types of documents that have been ascertained of the same size asdocument 11 is retrieved from master document file. As apparatus 10 isoperated at a specific site over time certain types of documents will beanalyzed more often than others. For example, if apparatus 10 is used tocheck driver licenses and identity cards at a state liquor store in theState of New Hampshire, the probability is that documents inserted intothe apparatus for verification will be New Hampshire driver licenses andidentity cards. Accordingly, over time the apparatus will collectinformation that most of the time New Hampshire driver licenses andidentity cards are ascertained and verified, and possibly driverslicenses more often than identity cards. Using this statisticalinformation, after a document size range determination is made for adocument, and the size range includes New Hampshire drivers licenses andidentity cards, the subsequent checking for the presence of informationat test regions on the document will initially look for the presence ofsuch information that is found first on New Hampshire drivers licensesand then on identity cards. If a document is not ascertained to be oneof these two types of documents, processing will continue for othertypes of documents in the size range. Thus, the processing time istypically shortened.

Using the information read in blocks 33 and 34, and following a searchordering set by the statistical information, at block 35 the informationfrom the first test region in the first picture is analyzed to determineif the information matches the retrieved characteristics for thatregion. If there is no very high confidence match the program progressesto block 36 and saves the match confidence determined.

If there is a very high confidence match, the program progresses toblock 37 where the image information in the second test region in thefirst picture is analyzed to determine if the information matches theretrieved set of characteristics for that test region. If there is novery high confidence match the program stores the match confidencedetermined at block 36. This process repeats through a number ofiterations as defined by a retrieved set of document definitions. Atblock 38 the digital image information for the last (Nth) region in thefirst picture is analyzed to determine if the information matches theset of characteristics for the Nth region. If there is no very highconfidence match the program again stores this determination at block36.

If there is a match the program progresses to block 39 where thedocument type is ascertained to a very high degree of confidence usingthe results of the image matching. Using the ascertained document typethe program progresses to block 45 and retrieves a document definitionfile that is used in analyzing and verifying the document as describedwith reference to FIG. 3.

At block 40 the program determines when the image matching is complete.When complete, at block 41 it is decided if any of the confidencereports from blocks 35, 37 and 38 exceed a predetermined minimumthreshold. If any do, then the reference documents highest 3(configurable number) scores are checked against document 11 using thesecondary characteristics and test regions as specified in theirrespective document definition file. The process is the same as steps36–40. This procedure is followed until it has either been determinedthat a document is a specific type to a very high degree of confidenceor the operator is notified that the type is questionable.

If none of the confidence reports exceeds the minimum threshold thismeans either document 11 on platen 12 is not placed thereon in thecorrect orientation, the document is in too poor a condition forrecognition, or the document type is not included in the master documentfile of apparatus 10. It is virtually impossible to mis-orient apassport, however, that will not be the usual case for the smaller IDcards. It is then determined at block 42 if the first image is not apassport-size and has already been electronically rotated one-hundredeighty degrees.

If the image has not been previously rotated, at block 43 the image iselectronically rotated one-hundred eighty degrees and the programreturns to block 35 and the analysis formed at blocks 35, 36, 37, 38,40, 41 and 42 is repeated. If the document type still cannot beascertained a message is displayed “Can't Identify Document Type”. Ifthe image has not previously been rotated, as determined at block 42, itis clearly determined that the document type is not included in thedatabase, and the program progresses to block 44 where a message isforwarded, “Can't Identify Document Type”. Thus, apparatus 10 willnotify all interested parties that other action must be taken to resolvethe matter.

If after electronically rotating the first picture, apparatus 10 canidentify the document, processing of the document continues using therotated image. Using the ascertained document type, the programprogresses to block 45 and retrieves a document definition file fromwhich inspection criteria are retrieved to be used in analyzing andverifying the document as described with reference to FIG. 3.

FIG. 3 is a block diagram showing the steps involved in reading andcapturing data from the document, and testing to determine if a documentis valid, counterfeit or has been altered. To simplify this figure theoperation is described with reference to three basic types of data to beobtained from documents. The first information type is obtained fromimages created using ultraviolet, infrared, direct, polarized light,etc., where one or more additional images are obtained and thenprocessed to derive data therefrom. The second information type ispictorial or graphic information on documents, such as photo,fingerprint, or seal, and is captured directly from the document image.The third information type is data read directly from an image on thedocument by optical character reading (OCR), intelligent characterrecognition (ICR), bar code interpretation, etc. In some instances datamay be derived from the other images of the document, when suchinformation becomes visible under a special light. There may be morethan one type of information in each of these data types, such as OCR,ICR and bar codes being on the same document. In addition, forinformation not found in any of the images, there may a magnetic striperead by a magnetic stripe reader, RF reader, ultrasonic reader or asmart card reader. Different steps shown in FIG. 3 occur in parallel sothe processing time is minimized.

Information in a standardized format is often located in fixedmachine-readable zones (“MRZ”) on a document such as established underICAO 9303 or ISO-7501 standards. If an MRZ is detected, CPU 14 uses anoptical character reading (OCR) technology to “read” the alphanumericinformation in the MRZ field. By analyzing information in the MRZ field,CPU 14 is able to determine if the document is a passport, or anothertype of document that includes an MRZ field. Such MRZ information mayinclude, but is not limited to, the name, birthday, sex, place of birthof the person to whom the document is issued, the date of issuance andexpiration of the document, the issuing authority, issue run, anddocument number. This information may also be encrypted and placed inbar codes or on microchips on documents, and used as a double checkagainst visible information to verify that a document is not a forgeryand/or has not been altered. In addition, laser readable material,invisible to the human eye, may be placed on the document which iswritten and read like a CDROM, but is written and read in rows, and maycontain data visible elsewhere on the card, or encoded fingerprints,hand-geometry, iris scans, retinal scans, and other biometricinformation.

Certain inks, such as those containing carbon black, absorb infraredlight. Other inks do not absorb infrared light, but are transparent toit or reflect it. Both may appear to be the same color under normallight.

Printing on a document is generally in black, but MRZ items on thedocument will be printed with the special near-IR absorbing (B900) blackink to conform to international standards. When illuminated with anear-IR source this latter printing will appear, while all otherprinting disappears. CPU 14 knows where to look for the special inkprinting in a digitized image made under illumination of the near-IRsource from the retrieved document definition file. If the near-IRresponsive images are in the specified areas, whether they arealphanumeric text or certain patterns or images, they will be identifiedby CPU 14 as an indication that the passport 11 in document reader 10has not been photocopied.

A long wave ultraviolet (UV-A) light causes certain inks and fibers tofluoresce, so they are visible within the image captured by camera 18using this light source. A short wave ultraviolet (UV-C) causes otherspecial inks and fibers to fluoresce, while all other areas on thedocument image disappear, including those made with materials thatfluoresce under UV-A. In addition, alphanumeric characters and symbolsmay be printed on passport 11 or other documents with inks that are notvisible to the human eye, but which appear when illuminated with a UVlight source. These symbols may be printed on the portion of thepassport that is laminated, or may be imprinted in or on the laminatingmaterial. From the retrieved document definition file CPU 14 knows whereto look in a digitized picture captured from a document for the symbolsthat are visible when illuminated under a UV light source.

Document definition files include document layout (location of differenttypes of information and pictures on a document), security features(such as information visible with ultraviolet, infrared, direct blue,and polarized light) and intrinsic characteristics (documetric) specificto the process or materials used to manufacture the document. Usingretrieved document definitions, only selected ones of the UV, IR andother images are captured, and only specific regions in the images willbe analyzed. In addition, images are obtained using infrared,ultraviolet or other wavelengths of light; or polarized, coaxial, orlight incident at specific angles at resolutions adapted to provide thebest analysis for the type of data being read. These additional imagesare processed to obtain further data on the document, for checking theauthenticity of the document, and for determining if the document hasbeen altered or is a counterfeit.

The reason that all possible images of a document are not capturedsequentially is that all documents do not include features that requireobtaining images using all of the combinations of excitation sources orcamera resolutions available. In this way only required images areobtained and the document processing time is minimized.

In FIG. 3, at block 51, first data is captured by taking an ultraviolet,infrared, polarized etc. picture of document 11 as established by thedocument definition file for that document type. Assume that at block 51an ultraviolet image is obtained first. At block 53 first datainformation made visible by the ultraviolet light is read and stored.Merely seeing if it is visible at specific locations on the document, orif the information made visible may be read using another technologysuch as optical character reading, barcode reading, or image recognitionmay check the information. At block 54 the information obtained usingthe ultraviolet picture, as described in the previous sentence, iscompared to reference information from the document definition file. Atblock 55 the weighted results of the comparison are saved.

At block 56 a decision is made if the result of the comparison of theinformation obtained using the ultraviolet image and the referenceinformation from the document definition file indicates the confidencelevel that there is a match or, conversely, indicates the possibilitythat the document is a counterfeit or has been altered.

At the same time that the ultraviolet picture is being read andanalyzed, a decision is made at block 52 if there are more first datapictures to be obtained and analyzed. If there are more first datapictures to be obtained the program branches back to the input of block51 to again capture a first data picture, such as an infrared picture tobe used, for example, to check for the use of certain carbon based inkson the document, or for alterations to the document. At block 53information made visible by the infrared light is read and saved. Merelyseeing if information is visible at specific locations on the document,or if the information made visible may be read using another technologysuch as optical character reading, and barcode reading may check theinformation. At block 54 the information obtained using the infraredpicture is compared to standard reference information from the documentdefinition file. At block 55 the weighted results of the comparison aresaved.

When the coaxial light sources are used in the detection andverification of retro-reflective materials contained in documents theuse of an image, if any, in the retro-reflective material may be viewed,and the image, if any, may be compared to a reference image from adocument definition file that is retrieved from storage, either locallyor remotely. One example of such retro-reflective materials is 3M'sConfirm material that is described in more detail in the followingparagraphs.

When 3M's Confirm laminate is illuminated with coaxial IR light theimage captured is a continuous gray and an embedded logo does notappear. It looks like a clean, gray slate. The continuous gray is easilydetected as an indication of the presence of the Confirm material. Anyalterations to and tampering with the Confirm laminate appear as blackmarks on the gray background and are easily detected. Further, atincreased power levels, the light is reflected from the bottom surfaceof the laminate or the surface of the document which is laminated in amanner that reveals the use of unauthorized laminates, and alterationsto the laminate. In addition, when viewing the Confirm material, IRcoaxial illumination is not affected by normal scuff marks andscratches, fingerprints and dirt on the surface of the laminate.

A direct blue light source generated by an array of blue LEDs isspecifically used to verify that 3M's retro-reflective Confirm® materialis used as the laminate, and has not been tampered with. Under this bluelight a white logo is seen against a gray background. This is easilydetected by analysis of the image produced, and can be read usingoptical character reading. Such logos are combinations of words andgraphics that are distinctive to the country or issuer of the passportor other type of document and are compared to words and graphics in thedocument definition file for the document. The logo is invisible to thenaked eye. Any attempts to forge the 3M laminate, or to use anotherlaminate, are obvious.

At the same time that the image is being read and analyzed, using thedocument definition file a decision is made at block 52 if there aremore first data pictures to be obtained and analyzed. If there are nomore first data pictures to be captured the program branches from block52 to block 58 were it is determined if there are any second data typepictures to be captured. If there are none, the program branchesdirectly to the input of block 62. When there are second data typepictures to be captured, per the document definition file, the programadvances to block 59 and a photo or other pictorial information on thedocument at a specifically defined location is taken from the firstpicture and, at block 60, is stored. Typically, it will be a photographon the document that is captured and saved for other purposes such ascomparison with the bearer of the document by an attendant or by acomputer using an image of the bearer's finger/or information in adatabase using facial recognition technology.

At the same time a photograph image or other pictorial information isbeing stored at block 60, using the document definition file adetermination is being made at block 61 if there is another second datatype picture to be captured, such as some graphic located on the firstimage of the document. If there is, the program cycles back to the inputof block 59 to image the graphic at its specifically defined location.At block 60 the graphic is stored.

In the event that there are no additional second data type pictures tobe captured, the program progresses to the input of block 62 where,using the document definition file, it is determined if there are anyNth data type information to be captured from the first picture, such asthe OCR of characters or reading a bar-code data, as indicated by thedocument definition file. When the decision made is that there are noNth data type information to be saved, the program branches from block62. Upon the decision being made at block 62 that there is Nth data typeinformation to be saved, the program branches to block 63 where the Nthdata type information at a specified location on the document iscaptured from the first image. At block 64 the information is read andstored. If the information is alphanumeric text it is read using OCR andthe information saved. If the information is in barcode form the barcodeis decoded and the information derived therefrom is saved.

At the same time that a first Nth data type information is beenretrieved from the first image, read and saved, at block 65 the programmakes a determination using the document definition file if there areany more Nth data information to be read and saved. If there is, theprogram branches from block 65 back to the input of block 63 to captureand read the Nth data information. For example, the first Nth datainformation is alphanumeric text that is read using OCR and the secondNth data information is a bar code that is read.

When there is no more Nth data information to be read and saved theprogram exits block 65. At this time all images have been captured, alldata read therefrom saved, and a decision now can be made as to thedocument validity based on the scoring to the degree to which thedocument matches the reference information in the document definitionfile.

In FIG. 4 is a block diagram showing the steps involved in furthervalidating a document and verifying the identity of the bearer bycomparing data derive from the document with data in databases,determining if the document bearer matches the biometrics on thedocument (typically the photograph), if the bearer is on a watch list,wanted list, or is on a prohibited entry list. At block 70 theinformation retrieved from the document in FIG. 3 is used to accessdatabases related to the type of document being verified as specified inthe document definition file or established in a master control file bythe organization managing the operation of the system.

At block 74 a determination is made if there is a match between the twosets of information. The database information being compared may be apositive or negative indication of the document validity or theiridentity. The results are scored and used as a part of the overallevaluation of the document and the bearer.

Completion of the match test between the sets of information causes theprogram to progress to block 75 where other databases are accessed todetermine, for example, if the bearer of the document is not in theissuing authority database, or is on a watch list of any type. While notshown in FIG. 4 these other database tests might include: (1) checkingif the bearer of the document is wanted for any crimes, (2) checking thedocument against lists of stolen documents, (3) analysis of datacombinations such as arrival on departure locations, (4) checking forprohibited entry individuals, and (5) matching suspected combinations oftravelers and travel patterns. If the bearer of the document is on awatch list or wanted for any crimes, the program branches to block 77 tonotify the appropriate authorities.

In FIG. 5 is shown a block diagram of the steps involved in comparingbiometric data on a document with that received directly from thedocument bearer. The purpose for this comparisons is to validate thebearer of the document and to check against biometric “watch lists”. Toimplement this comparison a camera, a fingerprint scanner, and/or aniris scanner, microphone, or signature capture device must be locatedwith apparatus 10. A decision is made at block 81 whether or not thereis a match. When there is a match the program progresses to block 86 todisplay a message that there is a biometric data match. The program thenreturns to its start in FIG. 1.

If there is no biometric data information, this determination is made atblock 81. The no match determination may be due to an altered orcounterfeit document, but may also be due to the fact that the biometricdata on the document (such as the photo) is unusable. At block 82 awarning message is given to the operator of apparatus 10 or theappropriate authorities. In addition, the program progresses to block 83to query the issuing authority of the document or other authorityrequesting that they or their appointed trust authority match the dataand biometric and return the results. The biometric data from thedocument and/or from the bearer may be used, along with data from thedocument, and are forwarded to the authority that will be performing thematch.

At block 84 the match results from the trust authority are checked todetermine if there is a data match. If there is no data match, theprogram progresses to block 85 where an alert is given to the operatorof apparatus 10 and/or the appropriate authorities. Upon it beingdetermined at block 84 that there is a match, the program progresses toblock 86 to display a message that there is a biometric data match. Theprogram then returns to its start in FIG. 1.

In addition to biometric data matching, biometric information retrieveddirectly from the document bearer and/or from the document may becompared against a biometric “watch list”. If it is determined that thedocument bearer is on a biometric “watchlist”, an alert message is givento the operator of apparatus 10 and/or the appropriate authorities.

At this point apparatus 10 has verified both a document and the bearerof the document, and has verified that the bearer has authorization toparticipate in an activity represented by the document. The programrunning apparatus 10 returns to its start state in FIG. 2 and is readyto check another document.

While what has been described herein is the preferred embodiment of theinvention, it will be understood by those skilled in the art thatnumerous changes may be made without departing from the spirit and scopeof the invention.

1. A method for automatically processing documents of many differentclasses and types that are randomly presented to first identify theclass of each document, then identify the type of document within anidentified class of documents, the method comprising the steps of: (a)capturing a complete representation of an entire document that ispresented to be identified; (b) determining a first characteristic foreach document presented to be identified using its completerepresentation captured in step (a), the first characteristic being usedto identify one class of document from another class of document; (c)retrieving a set of second characteristics for each document presentedto be identified whose complete representation is captured in step (a)and whose class of document is identified in step (b), the second set ofcharacteristics being used to identify the type of document from amongstthe class of documents identified in step (b); and (d) analyzingindividual characteristics from the second set of characteristicsretrieved in step (c) with characteristics actually in the completedocument representation captured in step (a) to identify the type ofdocument from amongst the class of documents identified in step (b);wherein the determination of a first characteristic performed in step(b) is to determine the size of an document being processed, alldocuments that may be identified and verified are divided into sizeranges and each class of documents includes all documents having thesame size, and wherein the second set of characteristics retrieved instep (c) include color patterns at specific locations on documents. 2.The method for automatically processing documents according to claim 1where identified documents are to be verified, and further comprisingthe steps of: (e) retrieving a set of reference information unique toeach type of document that is identified in step (d); and (f) analyzingeach document whose complete representation is captured in step (a)using the unique set of reference information retrieved in step (e) toverify if the type of document identified in step (d) is genuine,counterfeit, or has been altered.
 3. The method in accordance with claim2 further comprising the step of: (g) providing an indication that andocument is genuine, counterfeit or has been altered based upon theresults of analysis performed in step (f).
 4. The method forautomatically processing documents in accordance with claim 1 furthercomprising the steps of: (e) ordering all document types that areidentified in step (d) from the most commonly identified type ofdocuments to the least commonly identified type of documents and (f)selecting the retrieved characteristics from step (c) for use in step(d) starting with characteristics for the most commonly identifieddocument type and progressing to the least commonly identified documenttype.
 5. The method for automatically processing documents according toclaim 4 where identified documents are to be verified, and furthercomprising the steps of: (g) retrieving a set of reference informationunique to each type of document that is identified in step (d); and (h)analyzing each type of document identified in step (d) using the uniqueset of reference information retrieved in step (g) to verify if it isgenuine, counterfeit, or has been altered.
 6. A computer readable mediumcontaining executable instructions for automatically processingdocuments of many different classes and types that are randomlypresented to first identify the class of each document, then identifythe type of document within an identified class of documents, theexecutable program instructions comprising program instructions for: (a)capturing a complete representation of an entire document that ispresented to be identified; (b) determining a first characteristic foreach document presented to be identified using its completerepresentation captured in step (a), the first characteristic being usedto identify one class of document from another class of document; (c)retrieving a set of second characteristics for each document presentedto be being identified whose complete representation is captured in step(a) and whose class of document is identified in step (b), the secondset of characteristics being used to identify the type of document fromamongst the class of documents identified in step (b); (d) analyzingindividual characteristics from the second set of characteristicsretrieved in step (c) with characteristics actually in the completedocument representation captured in step (a) to identify the type ofdocument from amongst the class of documents identified in step (b);wherein the determination of a first characteristic performed in step(b) is to determine the size of a document being processed, alldocuments that may be identified and verified are divided into sizeranges and each class of documents includes all documents having thesame size, and wherein the second set of characteristics retrieved instep (c) include color patterns at specific locations on documents. 7.The computer readable medium executable instructions of claim 6 furthercomprising instructions for: (e) retrieving a set of referenceinformation unique to each type of document that is identified in step(d); and (f) analyzing each document whose complete representation iscaptured in step (a) using the unique set of reference informationretrieved in step (e) to verify if the type of document identified instep (d) is genuine, counterfeit, or has been altered.